Multi-class Classification in Big Data

The paper suggests the on-line multi-class classi er with a sublinear computational complexity relative to the number of training objects. The proposed approach is based on the combining of two-class probabilistic classi ers. Pairwise coupling is a popular multi-class classication method that combines all comparisons for each pair of classes. Unfortunately pairwise coupling su ers in many cases from incompatibility in that some regions of its input space the sum of probabilities are not equal to one. In this paper we propose the optimal approximation for probabilities in each point of object space. This paper proposes a new probabilistic interpretation of the Support Vector Machine for obtaining class probabilities. We show how the SVM can be viewed as a maximum likelihood estimate of a class of probabilistic models. As a computational method for big data we use the stochastic gradient descent approach minimizing directly the primal SVM objective. Unfortunately the hinge loss of the true SVM classi er did not allow to use SGD procedure for determining the classi er bias. In this paper we propose the piece-wise quadratic loss that helps to overcome this obstacle and gives an instrument to obtain the bias from SGD procedure.